We propose a general and unified framework to harness the abundant performance of current and future generation commodity graphics hardware (GPUs) for the purpose of tomographic reconstruction from projections. Preliminary results indicate that a performance improvement on the order of 1-2 magnitudes over traditional CPU-based approaches can be obtained. To proof and ensure the generality of our framework and approach, our proposal will address the reconstruction from a diverse set of raw data (such as kV X-rays, MV X-rays, and protons), with a diverse set of reconstruction algorithms (such as maximum likelihood algorithms, algebraic methods, and Feldkamp-style filtered backprojection), and within a diverse set of application scenarios (such as CT, SPECT, PET, and Proton CT). Starting from the basic reconstruction operators, we will model all dominant physical and algorithmic effects that occur in radiation-based tomography, such as depth weighting, detector geometric response, attenuation weighting, and scatter compensation, using implementations that map optimally to the graphics hardware and take full advantage of its computational architecture. The rapid speeds of reconstruction will also enable a new concept that we call Visual Reconstruction Steering (VRS). This VRS framework will consist of a visual interface in which users can build and interactively control reconstructions as they occur on the GPU, and in which they can visualize and assess the results in real time. Our research results, all software, and documentation will be disseminated via a dedicated website, www.Rapid-CT.org.